{
 "metadata": {
  "name": "",
  "signature": "sha256:62875ff0994965e627d373b54f578bf39185c4b735c2ae1085964f93f2c0c80a"
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "heading",
     "level": 1,
     "metadata": {},
     "source": [
      "The PlanOut interpreter"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "PlanOut experiments can be implemented in pure Python, but they may also be specified via a platform-independent serialization, which can be generated via the PlanOut compiler or through some user interface. Since interpreters can be written in any language, it is possible for serialized PlanOut code to executed on multiple kinds of devices and server platforms."
     ]
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "The PlanOut language"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The PlanOut language provides a few simple constructs for defining experiments. It looks a lot like JavaScript, e.g.,\n",
      "\n",
      "```\n",
      "x = uniformChoice(choices=['a','b'], unit=userid);\n",
      "y = bernoulliTrial(p=0.2, unit=[userid, albumid]);\n",
      "```\n",
      "\n",
      "More documentation about the PlanOut interpreter and language can be [found on the PlanOut homepage](http://facebook.github.io/planout/docs/planout-language.html). One thing to note is that operators in the PlanOut language are lowercase, whereas pure Python operators are objects and start with upper case.\n",
      "\n",
      "To compile this code, you might write it to a file (say, `myfile.planout`), and type something like:\n",
      "\n",
      "```\n",
      "node planout.js myfile.planout\n",
      "```\n",
      "\n",
      "`planout.js` can be found in the `compiler/` directory of the PlanOut Github repository."
     ]
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "Calling the interpreter from Python"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "We use this handy-dandy function to compile PlanOut scripts directly from IPython notebooks."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from os import popen, system\n",
      "import json\n",
      "\n",
      "def compile(s):\n",
      "    \"compile planout code using planout.js compiler\"\n",
      "    with open('tmp.planout','w') as f:\n",
      "        f.write(s)\n",
      "    # the error handling here could be better...\n",
      "    d = json.loads(popen('node ../../compiler/planout.js tmp.planout').read())\n",
      "    system('rm tmp.planout')\n",
      "    return d"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 1
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "Serialized PlanOut code"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The PlanOut language gets compiled into JSON, which can be loaded into Python as a dictionary and run by the interpreter."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Here is what PlanOut code looks like when it gets compiled into the serialization."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "serial = compile(\"\"\"\n",
      "  button_text = uniformChoice(choices=['Purchase','Buy'], unit=userid);\n",
      "  has_discount = bernoulliTrial(p=0.3, unit=userid);\n",
      "\"\"\")\n",
      "serial"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 2,
       "text": [
        "{u'op': u'seq',\n",
        " u'seq': [{u'op': u'set',\n",
        "   u'value': {u'choices': {u'op': u'array', u'values': [u'Purchase', u'Buy']},\n",
        "    u'op': u'uniformChoice',\n",
        "    u'unit': {u'op': u'get', u'var': u'userid'}},\n",
        "   u'var': u'button_text'},\n",
        "  {u'op': u'set',\n",
        "   u'value': {u'op': u'bernoulliTrial',\n",
        "    u'p': 0.3,\n",
        "    u'unit': {u'op': u'get', u'var': u'userid'}},\n",
        "   u'var': u'has_discount'}]}"
       ]
      }
     ],
     "prompt_number": 2
    },
    {
     "cell_type": "heading",
     "level": 4,
     "metadata": {},
     "source": [
      "Running serialized PlanOut code"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The `Interpreter` object executes a dictionary containing serialized PlanOut code. `Interpreter` has three required arguments:\n",
      " * A dictionary containing serialized PlanOut code\n",
      " * An experiment-level salt\n",
      " * Input data"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Let's run the serialized code above with userid 4."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from planout.interpreter import *\n",
      "\n",
      "i = Interpreter(serial, 'my_salt', {'userid':4})\n",
      "i.get_params()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 3,
       "text": [
        "{u'button_text': u'Purchase', u'has_discount': 0}"
       ]
      }
     ],
     "prompt_number": 3
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "And with a few more users..."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "for i in xrange(5):\n",
      "    print i, Interpreter(serial, 'my_salt', {'userid':i}).get_params()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "0 {u'button_text': u'Buy', u'has_discount': 0}\n",
        "1 {u'button_text': u'Buy', u'has_discount': 0}\n",
        "2 {u'button_text': u'Purchase', u'has_discount': 0}\n",
        "3 {u'button_text': u'Purchase', u'has_discount': 0}\n",
        "4 {u'button_text': u'Purchase', u'has_discount': 0}\n"
       ]
      }
     ],
     "prompt_number": 4
    },
    {
     "cell_type": "heading",
     "level": 4,
     "metadata": {},
     "source": [
      "Serialized PlanOut code acts just like pure Python experiments"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Given that you use the same experiment-level and parameter-level salts, random assignment is completely deterministic. We can replicate the same assignments using the `SimpleExperiment` class."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from planout.experiment import SimpleExperiment\n",
      "from planout.ops.random import *\n",
      "\n",
      "class Doppelganger(SimpleExperiment):\n",
      "    def setup(self):\n",
      "        self.salt = 'my_salt'  # same salt as above\n",
      "    \n",
      "    def assign(self, params, userid):\n",
      "        params.button_text = UniformChoice(choices=['Purchase','Buy'], unit=userid);\n",
      "        params.has_discount = BernoulliTrial(p=0.3, unit=userid);\n",
      "\n",
      "for i in xrange(5):\n",
      "    print i, Doppelganger(userid=i).get_params()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "Running `SimpleExperiment`s with serialized PlanOut code"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "We define an abstract class `SimpleDictExperiment`, which inherets from `SimpleExperiment` but replaces the work done by `assign()` with serialized PlanOut code, which gets stored in class variable `script`."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import hashlib\n",
      "from abc import ABCMeta\n",
      "\n",
      "class SimpleDictExperiment(SimpleExperiment):\n",
      "  \"\"\"Simple class for loading a dictionary-based PlanOut interpreter experiment\"\"\"\n",
      "  __metaclass__ = ABCMeta\n",
      "  script = None\n",
      "\n",
      "  def assign(self, params, **kwargs):\n",
      "    i = Interpreter(\n",
      "      self.script,\n",
      "      self.salt,\n",
      "      kwargs\n",
      "      )\n",
      "    # this sets all of params' attributes to the key-value pairs in i\n",
      "    params.update(i.get_params())\n",
      "\n",
      "  def checksum(self):\n",
      "    # we log a checksum of the script so that analysts know if the experiment definition\n",
      "    # changed in some way.\n",
      "    src = json.dumps(self.script)\n",
      "    return hashlib.sha1(src).hexdigest()[:8]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "We can then subclass `SimpleDictExperiment`."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "class IExp1(SimpleDictExperiment):\n",
      "    def setup(self):\n",
      "        self.salt = 'my_salt'\n",
      "\n",
      "    script = compile(\"\"\"\n",
      "      button_text = uniformChoice(choices=['Purchase','Buy'], unit=userid);\n",
      "      has_discount = bernoulliTrial(p=0.3, unit=userid);\n",
      "    \"\"\")\n",
      "\n",
      "for i in xrange(5):\n",
      "    e = IExp1(userid=i)\n",
      "    print i, e.get_params()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "This is probably pretty close to something you'd like to use in a production setting."
     ]
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "Other ways of generating serialized experiments"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Serialized PlanOut code can also be generated automatically, including via user interfaces.\n",
      "\n",
      "Let's consider a hypothetical graphical user interface for constructing full factorial experiments.  We assume that the users' Web front end makes an AJAX request to the server, and sends a dictionary whose keys are factors (parameters) and values are lists of possible values.\n",
      "\n",
      "We would then use a function like this to generate PlanOut code for the given input data"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "def serialize_full_factorial(x,uid='userid'):\n",
      "    items = []\n",
      "    for k,v in x.iteritems():\n",
      "        items.append({\"op\":\"set\",\"var\":k, \"value\":\n",
      "          {\"op\":\"uniformChoice\", \"choices\":v, \"unit\":\n",
      "           {\"op\": \"get\", \"var\":uid}}})\n",
      "    return {\"op\":\"seq\", \"seq\": items}"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Here is an example configuration file that might be generated by the GUI, and how it gets transformed into PlanOut code."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "my_config = {'button_text':['Post', 'Share', 'X'], 'button_color':['#00ff00', '#aaaaaa']}\n",
      "serialize_full_factorial(my_config)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "We can generate experiments on the fly from the configuration file:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "class ExperimentStub(SimpleDictExperiment):\n",
      "    script = None\n",
      "    \n",
      "def gen_exp(experiment_name, config, **kwargs):\n",
      "    e = ExperimentStub(**kwargs)\n",
      "    e.name = experiment_name\n",
      "    e.salt = experiment_name\n",
      "    e.script = serialize_full_factorial(config)\n",
      "    return e"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Now let's perform some random assignments. You can also see the log file, `my-generated-experiment.log`."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "for i in xrange(5):\n",
      "    print i, gen_exp('my generated experiment', my_config, userid=i).get_params()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    }
   ],
   "metadata": {}
  }
 ]
}